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- the MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 



2a)D This action is FINAL. 2b)^ This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 
Disposition of Claims 

4) ^3 Claim(s) 1-21 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) [EI Claim(s) 1-14 & 18-21 is/are rejected. 

7) E3 Claim(s) 15-17 is/are objected to. 

8) Q Claim(s) are subject to restriction and/or election requirement 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) ^ The drawing(s) filed on 01 November 2000 is/are: a)^ accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

1 1) D The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§119 and 120 

13) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 

a)QAII b)D Some*c)D None of: 

1. Q Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1) Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) O Notice of Informal Patent Application (PTO-152) 

3) £3 Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 
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DETAILED ACTION 
Claim Objections 

1. Claims 7-9 & 14-17 are objected to because of the following informalities: Close 
parenthesis is missing. Appropriate correction is required. 

Claim Rejections - 35 USC §102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

2. Claims 1-8 & 18-21 are rejected under 35 U.S.C. 102(e) as being anticipated by Brill et al 
(U.S. Patent No. 6,542,621 Bl). 

a. Referring to Claim 1, Brill discloses processing a sequence of images (Brill at 
column 4, lines 59-60, discloses processing a succession of images provided by the video 
camera 12) to generate a statistical model (Brill at column 6, lines 5-8, teaches 
probabilistic templates or p-templates. The path of a p-template through the scene 
represents the track of a given person, which is maintained by the tracking system. Such 
probabilistic template is the claimed "statistical model") for each person to be tagged 
(Brill at column 6, lines 10-11, teaches the p-template is assigned to each of tracked 
objects in the video sequence. Note the objects herein are people as shown in figures 6A- 
6D), the statistical model incorporating at least one appearance feature (Brill at column 8, 
lines 22-23 discloses a p-template includes a raw color corrected probability. Note color 
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is an appearance feature) and at least one geometric feature (Brill at column 7, lines 37- 
39, teaches the p-template also depends on the location of the tracked object. The location 
herein corresponds to the claimed "geometric feature") of the tagged person; applying the 
model to at least one subsequent image in order to perform at least one of a detection 
operation, a location operator and a tracking operation for the tagged person (Brill at 
column 8, line 34, teaches a p-template is applied for each tracked object, which implies 
the p-template is applied in a tracking operation); and controlling an action of the image 
processing system based on a result of the at least one operation (Brill at column 1 , line 
41, teaches applying such tracking technique in physical security environment, such as 
surveillance camera , which the tracking algorithm will direct the video camera to track 
the given objects). 

b. Referring to Claim 2, Brill discloses wherein the sequence of image comprises a 
video segment (column 4, lines 59-60). 

c. Referring to Claim 3, Brill discloses wherein the processing step further includes 
processing the sequence of images to generate a plurality of statistical models, each of the 
models corresponding to a particular tagged person (Brill at column 6, lines 10-11 
teaches using plurality of p-templates when tracking objects in video sequence. Brill at 
column 8, line 34 teaches such p-template is generated for each tracked object). 

d. Referring to Claim 4, Brill discloses wherein the appearance feature comprises at 
least one of a color feature and a texture feature (Brill at column 8, lines 23-24, discloses 
color feature). 
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e. Referring to Claim 5, Brill discloses wherein the geometric feature comprises at 
least one of a region shape and a region position for a given one of a plurality of regions 
associated with the statistical model (Brill at column 7, lines 37-38, teaches the p- 
template depends on object locations, also see column 5, lines 64-67, and column 6, lines 
1-2). 

f Referring to Claim 6, Brill discloses wherein the statistical model is generated at 
least in part by segmenting a given image into a number N of different regions of similar 
appearance (Brill teaches at column 6, lines 10-11 teaches applying 2 p-templates, and 
they are segmented in different regions in a video image as shown in figure 5. And they 
have similar appearance, both are ovals). 

g. Referring to Claim 7, Brill discloses wherein the statistical model generated for a 
given person comprises a likelihood probability function which indicates the likelihood 
that the person is presented in a given image (Brill teaches generated p-template for a 
given object in a given video sequence, see column 6, lines 5-17) 

h. Referring to Claim 8, Brill discloses wherein the likelihood probability function 
P(I | Q) for person Q is computed as P(I | Q) = ^P(R r \ 0)P(r | Q) (Brill at column 

9, line 56, discloses use of the Bayesian probabilistic framework), where R r is a function 
of the at least one appearance feature and the at least one geometric feature (Brill teaches 
use of the appearance feature for p-template), and r is an index identifying one of N 
regions of similar appearance within the image I (Brill teaches calculating the p-template 
for a given object in a given video image). 
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i. Referring to Claim 18, Brill discloses wherein the controlling step comprises 
generating an output of the image processing system based on the result of the at least 
one operation (see explanation in claim 1). 

j. Referring to Claim 19, Brill discloses wherein the controlling step comprises 
altering an operating parameter of the image processing system based on the result of the 
at least one operation (Brill at column 9, lines 48-51, teaches when a new person enters 
the scene or a person leaves, the tracking algorithm will alter some parameter in the 
tracking operation). 

k. With regard to Claim 20, all the limitations are addressed in Claim 1 . 
1. With regard to Claim 21, the only difference between Claim 21 and Claim 1 is 
Claim 21 calls for an article of manufacture comprising a storage medium for storing one 
or more programs. Brill at column 3, line 25 teaches using a computer workstation 13 to 
perform image processing, which inherently includes a storage medium for storing one or 
more programs. 

Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3. Claims 9-14 are rejected under 35 U.S.C. 103(a) as being unpatentable over Brill in view 

of Chang et al (U.S. Patent No. 5,999,651), all the arguments and applicability in Claim 1 are 

incorporated herein. 
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a. Referring to Claim 9, Brill discloses so-called statistical model generated for a 



given person comprises a likelihood probability function, which contains motion 
values as parameters. (Brill: column 3, lines 36-54 and column 7, lines 27-31) 
However, Brill does not explicitly distinguish the use of global motion and 
location motion. Chang at column 4, lines 17-37, teaches obtaining first contour 
model using global motion estimation, and at column 5, lines 13-20, teaches 
performing location motion estimation based on the first contour model to obtain 
a second contour model. Therefore, the estimate probability described at column 
6, line 45, is a probability function based on global motion parameter and location 
motion parameter, which corresponds to the claimed "probability function" 
P(7|r,£,Q). In addition, Chang at column 4, line 28, teaches 



14 is the claimed "a discrete variable". At the time the invention was made, it 
would have been obvious to a person of ordinary skill in the art to apply the 
probability estimation technique taught by Chang in Brill's system. One of 
ordinary skill in the art would have been motivated to do this because Chang and 
Brill both provide teachings for object tracking (even though Brill performs 
tracking on people, and Chang gives an example of tracking a bird on a lake, 
Chang does point out at column 1, line 19 that his system is also applicable to 
human), and using probability estimation to locate the object movements. 
Additionally, Chang provides local motion probability to detect the object 
movements more precisely, which reduces the error estimation rate. 



d t {t)=d t (t-\) 



as the claimed "linear transformation", v,.(f)at column 5, line 
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b. Referring to Claim 10, Chang discloses wherein a location of the person is 
determined using the linear transformation T (Chang at column 4, line 28 teaches 
d ( / ) = d ( t - 1 ) as * e li near transformation). 

c. Referring to Claim 11, Chang discloses wherein a pose of the person is 
determined using the discrete variable £ (Chang at column 5, line 14 teaches v^t) 

as the discrete variable). 

d. Referring to Claim 12, Chang discloses wherein the linear transformation T is 
used to obtain a sub-window of the image I that is invariant to rotation and scale 
(Chang discloses use of snake tracking, which is inherently invariant to rotation 
and scale). 

e. Referring to Claim 13, Chang discloses wherein the linear transformation T is 
implemented using a bilinear interpolation technique with a reference point x c , a 

rotation angle 0, and a scaling factor s (Chang discloses reference point at 
column 4, line 25, the rotation angle and scaling factor are inherently included in 
snake tracking algorithm). 

f. Referring to Claim 14, Chang discloses wherein the local motion is modeled 
using a discrete set of states of the variable £ to capture M different poses of the 
person (Chang at column 5, line 14, discloses snake point v, (f)). 

Allowable Subject Matter 
4. Claims 15-17 are objected to as being dependent upon a rejected base claim, but would 
be allowable if rewritten in independent form including all of the limitations of the base claim 
and any intervening claims. 
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Reasons for allowance: 

a. Claim 1 5 defines a statistical model generated for a given person Q and image I 
comprises a likelihood probability function P(I | r,£,Q) = £P(p« | T,£,Cl) , 

pixel 

where r is an index to regions of similar appearance and TV is a total number of 
such regions, r=l,2,...N, and 

P(pix | r, 7\ Q) = max[P(pix \ r, T 9 £ Q)P(r |£,Q)] , where | r, 7, £ Q) is 

the probability of observing pixel pix assuming that it belongs to an r-th region of 
the model on a pose £ , and P(r \ Q) is the prior probability of the region at that 
pose. 

b. Claims 16-17 are dependent upon Claim 15. 

Conclusion 

5. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

a. Crabtree et al 3 U.S. Patent No. 6,185,314 Bl, see column 8, figure 8. 

b. Lu et al, U.S. Patent No. 5.550.928, see columns 4-7, figure 4. 

c. Darrel et al, U.S. Patent No. 6,188,777 Bl, see column 2. 

d. Comaniciu et al, U.S. Patent No. 6,590,999 Bl, see column 1, lines 42-67, column 
2, lines 1-5. 

6. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tom Y Lu whose telephone number is (703) 306-4057. The 
examiner can normally be reached on 8:30AM-5PM. 




Application/Control Number; 09/703,423 



Page 9 



Art Unit: 2621 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Leo H Boudreau can be reached on (703) 305-4706. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 



Tom Y. Lu 
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